Random mappings designed for commercial search engines

We give a practical random mapping that takes any set of documents represented as vectors inEuclidean space and then maps them to a sparse subset of the Hamming cube while retaining ordering ofinter-vector inner products. Once represented in the sparse space, it is natural to index documents usingcommercial text-based search engines which are specialized to take advantage of this sparse and discretestructure for large-scale document retrieval. We give a theoretical analysis of the mapping scheme,characterizing exact asymptotic behavior and also giving non-asymptotic bounds which we verify throughnumerical simulations. We balance the theoretical treatment with several practical considerations; theseallow substantial speed up of the method. We further illustrate the use of this method on search overtwo real data sets: a corpus of images represented by their color histograms, and a corpus of daily stockmarket index values.